Alternating Direction Method Of Multipliers Research Articles

Channel Estimation for Underwater Acoustic Communications in Impulsive Noise Environments: A Sparse, Robust, and Efficient Alternating Direction Method of Multipliers-Based Approach

Channel estimation in Underwater Acoustic Communication (UAC) faces significant challenges due to the non-Gaussian, impulsive noise in ocean environments and the inherent high dimensionality of the estimation task. This paper introduces a robust channel estimation algorithm by solving an l1−l1 optimization problem via the Alternating Direction Method of Multipliers (ADMM), effectively exploiting channel sparsity and addressing impulsive noise outliers. A non-monotone backtracking line search strategy is also developed to improve the convergence behavior. The proposed algorithm is low in complexity and has robust performance. Simulation results show that it exhibits a small performance deterioration of less than 1 dB for Channel Impulse Response (CIR) estimation in impulsive noise environments, nearly matching its performance under Additive White Gaussian Noise (AWGN) conditions. For Delay-Doppler (DD) doubly spread channel estimation, it maintains Bit Error Rate (BER) performance comparable to using ground truth channel information in both AWGN and impulsive noise environments. At-sea experimental validations for channel estimation in Orthogonal Frequency Division Multiplexing (OFDM) systems further underscore the fast convergence speed and high estimation accuracy of the proposed method.

Distributed Optimization of Multi-Microgrid Integrated Energy System with Coordinated Control of Energy Storage and Carbon Emissions

The mutual optimization of a multi-microgrid integrated energy system (MMIES) can effectively improve the overall economic and environmental benefits, contributing to sustainability. Targeting a scenario in which an MMIES is connected to the same node, an energy storage coordination control strategy and carbon emissions management strategy are proposed, and an adaptive step-size method is applied to improve the distributed optimization of MMIESs based on the alternating direction multiplier method (ADMM). Firstly, the basic framework of MMIESs is established, and a coordinated control strategy limiting the time of charge and the discharge of the battery storage system (BSS) is proposed. Then a multi-objective optimization model based on operating and environmental cost is formulated. Considering that different microgrids may be managed by different operators and a different convergence speed of multi-objective optimization iteration, an adaptive step-size distributed iterative optimization method based on ADMM is used, which can effectively reduce the cost and protect the privacy of each microgrid. Finally, a system composed of three microgrids is taken as an example for simulation analysis. The results of distributed optimization are accurate, and the proposed coordinated control strategy can effectively enhance the revenue of ESS, which verifies the effectiveness of the proposed method.

Fully Distributed Economic Dispatch with Random Wind Power Using Parallel and Finite-Step Consensus-Based ADMM

In this paper, a fully distributed strategy for the economic dispatch problem (EDP) in the smart grid is proposed. The economic dispatch model considers both traditional thermal generators and wind turbines (WTs), integrating generation costs, carbon trading expenses, and the expected costs associated with the unpredictability of wind power. The EDP is transformed into an equivalent optimization problem with only an equality constraint and thus can be solved by an alternating-direction method of multipliers (ADMM). Then, to tackle this problem in a distributed manner, the outer-layer framework of the proposed strategy adopts a parallel ADMM, where different variables can be calculated simultaneously. And the inner-layer framework adopts a finite-step consensus algorithm. Convergence to the optimal solution is achieved within a finite number of communication iterations, which depends on the scale of the communication network. In addition, leveraging local and neighbor information, a distributed algorithm is designed to compute the eigenvalues of the Laplacian matrix essential for the finite-step algorithm. Finally, several numerical examples are presented to verify the correctness and effectiveness of the proposed strategy.

SSCNet: learning-based subspace clustering

Sparse subspace clustering (SSC), a seminal clustering method, has demonstrated remarkable performance by effectively solving the data sparsity problem. However, it is not without its limitations. Key among these is the difficulty of incremental learning with the original SSC, accompanied by a computationally demanding recalculation process that constrains its scalability to large datasets. Moreover, the conventional SSC framework considers dictionary construction, affinity matrix learning and clustering as separate stages, potentially leading to suboptimal dictionaries and affinity matrices for clustering. To address these challenges, we present a novel clustering approach, called SSCNet, which leverages differentiable programming. Specifically, we redefine and generalize the optimization procedure of the linearized alternating direction method of multipliers (ADMM), framing it as a multi-block deep neural network, where each block corresponds to a linearized ADMM iteration step. This reformulation is used to address the SSC problem. We then use a shallow spectral embedding network as an unambiguous and differentiable module to approximate the eigenvalue decomposition. Finally, we incorporate a self-supervised structure to mitigate the non-differentiability inherent in k-means to achieve the final clustering results. In essence, we assign unique objectives to different modules and jointly optimize all module parameters using stochastic gradient descent. Due to the high efficiency of the optimization process, SSCNet can be easily applied to large-scale datasets. Experimental evaluations on several benchmarks confirm that our method outperforms traditional state-of-the-art approaches.

Distributed optimal Volt/Var control in power electronics dominated AC/DC hybrid distribution network

AbstractThe integration of large‐scale distributed power sources increases the voltage fluctuation in AC/DC hybrid distribution network (AD‐HDN). Power electronics devices such as photovoltaic (PV) inverters, soft open point (SOP), and voltage source converters (VSCs) can be utilized for voltage/var control (VVC) to alleviate the risk of voltage fluctuation and violation. This paper proposes a distributed optimal VVC method in power electronics dominated AD‐HDN. Firstly, the reactive power and voltage characteristics of PV inverters, SOP, and VSCs are analysed, and an optimal VVC optimization model for AD‐HDN to minimize node voltage deviation, PV curtailment, and network loss is proposed. Then, the second‐order cone (SOC) relaxation technique is used to re‐formulate the model into a convex optimization model. A distributed optimal VVC framework based on the alternating direction method of multipliers (ADMM) is constructed. Based on the residual balance principle and relaxation technique, an accelerated ADMM method is further proposed to solve the proposed model. Finally, case studies are conducted on the IEEE 33‐node and 85‐node systems to verify the superiority and effectiveness of the proposed method.

A branch-and-price algorithm for integrating urban rail crew scheduling and rostering problems

Crew planning, involving how to best schedule crew members during a given period, is a significant problem for urban rail transit companies. This paper proposes a new integer linear programming (ILP) model that can simultaneously optimize urban rail crew scheduling and rostering problems. The proposed ILP model is a set partitioning-based model with only one type of important duty selection variable that connects the two-level problem and circumvents the drawbacks of conventional approaches that usually formulate the crew scheduling and rostering problems separately and couple these two problems through linking constraints. This study demonstrates that the structure of the underlying network used to model the problem enables the development of an effective, heuristic branch-and-price procedure. The study compares the proposed approach with two other decomposition methods, namely Lagrangian relaxation and alternating direction method of multipliers (ADMM), on problems of different sizes and shows that the method provides lower bounds that are on average 16.4% better than Lagrangian relaxation and 5.03% better than ADMM, respectively. Furthermore, the study shows that, with an average optimality gap of 3.28%, the proposed approach obtains high-quality integer solutions to the integrated problem.

Distributed predictive motion planning of automated guided vehicles: serial versus parallel schemes

The article presents two distributed motion planning methods for dynamically coordinating multiple automated guide vehicles (AGVs) in an industrial setting, aiming to enhance their flexibility, robustness and scalability. A predictive motion model is utilized to describe the transport process mathematically as a dynamical system. Subsequently, alternating direction method of multipliers (ADMM)-based decomposition techniques, both serial and parallel, are developed to coordinate the AGVs and mitigate the computational burden. The efficacy of the distributed methods is verified through testing in industrial scenarios, demonstrating that they can significantly improve multi-AGV system transport productivity with minimal computational effort. Notably, the parallel scheme exhibits superior performance in coordinating the AGVs compared to the serial scheme.

Network alternating direction method of multipliers for ultrahigh‐dimensional decentralised federated learning

Ultrahigh‐dimensional data analysis has received great achievement in recent years. When the data are stored in multiple clients and the clients can be connected only with each other through a network structure, the implementation of ultrahigh‐dimensional analysis can be numerically challenging or even infeasible. In this work, we study decentralised federated learning for ultrahigh‐dimensional data analysis, where the parameters of interest are estimated via a large amount of devices without data sharing by a network structure. In the local machines, each parallel runs gradient ascent to obtain estimators via the sparsity‐restricted constrained methods. Also, we obtain a global model by aggregating each machine's information via an alternating direction method of multipliers (ADMM) using a concave pairwise fusion penalty between different machines through a network structure. The proposed method can mitigate privacy risks from traditional machine learning, recover the sparsity and provide estimates of all regression coefficients simultaneously. Under mild conditions, we show the convergence and estimation consistency of our method. The promising performance of the method is supported by both simulated and real data examples.

Adaptive orthogonal semi-supervised feature selection with reliable label matrix learning

Feature selection (FS) can select features with high value from high-dimensional data as much as possible and reduce the dimensionality of the data to improve the performance of the machine learning model and enhance the generalization ability. In semi-supervised FS (SSFS), regression-based methods utilize the label matrix, and the quality of the label matrix can directly affect the performance of FS. Hence, the importance of the reliable label matrix is obvious. Orthogonal regression (OR) can retain more information in the subspace than least squares regression (LSR). Therefore, this paper introduces OR into SSFS, and then the label scaling technique is used to learn a reliable label matrix. Also, this study utilizes adaptive graph learning to exploit more structural information about the data. Two constraints, Frobenius-norm or maximum information entropy imposed on the similarity matrix and two adaptive orthogonal SSFS (AGLSOFS) methods with reliable label matrix learning are constructed. The impact of these two constraints on the construction of dynamic similarity graphs and FS results is discussed. Effective optimization algorithms for these two methods are based on the Alternating Direction Method of Multipliers (ADMM) and Generalized Power Iteration (GPI). Experiments are conducted on 15 benchmark datasets, and the results show that: (1) similarity graphs constructed using both original and projected data are more accurate; (2) both constraints are valid; (3) both of the methods of this paper perform well on most datasets; (4) OR performs better than LSR for FS; and (5) the scaling factor affects the convergence speed of the model.

Efficient Dynamic Distributed Resource Slicing in 6G Multi-Access Edge Computing Networks With Online ADMM and Message Passing Graph Neural Networks

We consider the problem of resource slicing in the 6 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> generation multi-access edge computing (6G-MEC) network. The network includes many non-stationary space-air-ground-sea nodes with dynamic, unstable connections and resources, where any node can be in one of two hidden states: i) reliable – when the node generates/propagates no data errors; ii) unreliable – when the node can generate/propagate random errors. We show that solving this problem is challenging, since it represents a non-deterministic polynomial-time (NP) hard dynamic combinatorial optimization problem depending on the unknown distribution of hidden nodes' states and time-varying parameters (connections and resources of nodes) which can only be observed locally. To tackle these challenges, we develop a new deep learning (DL) model based on the message passing graph neural network (MPNN) to estimate hidden nodes' states in dynamic network environments. We then propose a novel algorithm based on the integration of MPNN-based DL and online alternating direction method of multipliers (ADMM) – extension of the well-known classical “static” ADMM to dynamic settings, where the slicing problem is solved distributedly, in real time, based on local information. We prove that our algorithm converges to a global optimum of our problem with a superior performance even in the highly-dynamic, unreliable scenarios.

Sparse L0-norm least squares support vector machine with feature selection

Least squares support vector machine (LSSVM) is a powerful classification tool based on hyperplanes. But the classical LSSVM does not perform well on small sample size data sets (SSS) because it lacks feature selection capability. To address this issue, we introduce an L0-norm regularization term into the objective function of LSSVM, and propose a new sparse LSSVM model called L0-LSSVM. However, the introduction of the L0-norm makes the resulting optimization problem nonconvex and nonsmooth. To overcome these challenges, instead of using the traditional convex or non-convex approximation of the L0-norm, we design an efficient algorithm for L0-LSSVM based on alternating direction method of multipliers (ADMM) where the L0-norm is handled by employing the proximity operator. Numerical results demonstrate that L0-LSSVM not only exhibits excellent generalization performance and feature selection capability but also offers a substantial improvement in computational speed.

Grouped Multivariate Variational Mode Decomposition With Application to EEG Analysis.

In this paper, a novel extended form of multivariate variational mode decomposition (MVMD) method to multigroup data named as grouped MVMD (GMVMD) is proposed. GMVMD is distinct from MVMD as it extracts common frequencies with strong correlations among regional channels. Firstly, GMVMD utilizes a new clustering algorithm named as frequencies grouping algorithm to classify the nearest common frequencies among all channels to specified groups. Secondly, a generic variational optimization model which is extended from MVMD is formulated. Thirdly, alternating direction method of multipliers (ADMM) is utilized to obtain optimal solution of GMVMD model. The proposed method introduces an extra parameter to decide the number of clusterings which need to be specified by the user. The effectiveness and superiority of the algorithm are demonstrated on a series of experiments. The utility of GMVMD is verified by grouping real-world electroencephalogram (EEG) data having similar center frequencies successfully. GMVMD outperforms MVMD in mode-alignment, signal reduction error and et al. Significance: GMVMD can obtain more accurate center frequencies and less signal reduction error than MVMD.

Robust condition identification against label noise in industrial processes based on trusted connection dictionary learning

In the era of big data, the pervasive use of artificial intelligence (AI) technology has revolutionized various industries. AI-powered systems, particularly those utilizing data-driven interpretable and reliable methods for accurate condition identification, have emerged as a crucial factor in achieving intelligent industrial systems. However, in real industrial sites, the labels of offline data often contain incorrect labels, i.e. label noise, due to human error, sensor failure, and other sources, which reduces the accuracy of the model and affects the performance and safety of the industrial system. To tackle the challenge of precarious performance caused by label noise, this paper proposes a robust method called Trusted Connection Dictionary Learning (TCDL) which effectively mitigates the adverse effects of label noise through interpretable constraints and guarantees the trustworthiness of the model for industrial applications. Specifically, to address the issue of data structure and label inconsistency arising from label noise, this paper presents a dynamic graph model that jointly optimizes data features and label information to obtain consistent representations of them. Subsequently, to effectively fit the labeled data, TCDL leverages the label information from the nearest neighbors through the optimized graph structure and facilitates the self-correction of label noise, leading to robust representations of process features and a reliable condition classifier. Finally, a novel iterative optimization algorithm that combines the Block Coordinate Descent (BCD) method and the Alternating Direction Multiplier Method (ADMM) is also developed to ensure the convergence of the algorithm. Due to simple constraints, it can obtain accurately optimized variables with few iterations. Numerical simulations and industrial experiments demonstrate the superiority of the proposed method compared to some state-of-the-art methods.

Joint optimization for computation offloading and 3C resource allocations over wireless-powered and NOMA-enabled multi-access MEC

Multi-access mobile edge computing (MEC) enables mobile users (MUs) to offload tasks to proximal multiple MEC servers for fast task processing. Since MUs generally have stringent delay requirements and limited energy and MEC servers have finite communication, computation, and caching (3C) resources, the joint co-design and optimization over computation offloading and 3C resource allocation for wireless-powered multi-access MEC systems have been highly demanded, while having not been well studied yet. We consider a wireless-powered multi-access MEC, where multiple energy harvesting (EH) based MUs and multiple base stations (BSs) each equipped with an MEC server coexist. Each MU first harvests energy from nearby MEC servers, and then offloads its sub-tasks to multiple MEC servers concurrently based on non-orthogonal multiple access (NOMA), so as to reduce data offloading delay. We first formulate a delay minimization problem, by jointly optimizing MUs’ computation offloading, MEC servers’ computation and caching resources allocation, and system communication resources allocation. Then we propose an alternating direction multiplier method (ADMM) based distributed scheme to decompose the formulated optimization problem into several sub-problems, and use the block coordinate descending (BCD) method and the successive convex approximation (SCA) method to transform all sub-problems each corresponding to one MEC server to convex subproblems. Finally, we validate and evaluate our proposed scheme through numerical analyses, which show that our proposed distributed scheme can greatly reduce the min–max delay of all MUs by allowing each MU’s concurrent data offloading to multiple MEC servers using NOMA and by jointly optimizing computation offloading and 3C resource allocations. Also, the delay imposed by our proposed scheme is much smaller than that imposed by the Interior Point method, which shows the effectiveness of our proposed scheme.

Digital Pathology Image Reconstruction with Alternating Direction Method of Multipliers using Wavelet, Contourlet and Shearlet Transforms

Digital pathology refers to image-based environment in which acquisition, extraction and interpretation of pathology information is supported by computational techniques. It has a huge potential to facilitate the diagnostic process, however, big data size and necessity of large storage areas are challenging. Therefore, in this research, Compressed Sensing (CS) scheme is studied with digital pathology images in order to reduce the amount of data for reconstruction. CS requires the sparsity of signals for a successful recovery which means that different sparsifying bases can alter the final performance. Wavelet, Contourlet and Shearlet Transforms are investigated to sparsify the digital pathology images, it is seen that Contourlet Transform is superior. Alternating Direction Method of Multipliers (ADMM) is chosen for reconstruction since it is a robust and fast convex optimization method. Despite the fact that digital pathology images are less sparse than classical images, CS reconstruction is satisfactory, which emphasizes the potential of CS for digital pathology. This study can be pioneering in the field of CS with digital pathology so it can encourage further studies of CS based imaging with different type of microscopes or at different wavelengths.

Joint flexibility-risk managed distributed energy trading considering network constraints and uncertainty

This paper addresses the operational challenges in distribution systems caused by the integration of fluctuating renewable energy sources and rising peak loads from widespread electrification. We propose a new distributed energy coordination framework that facilitates seamless interoperation between peer-to-peer (P2P) energy trading and the operation of distributed flexibility resources. This framework ensures network constraints are respected while promoting Energy Trading Consistency (ETC). Our approach utilizes a two-stage dynamic energy and flexibility sharing mechanism. It achieves energy efficiency through demand-management and P2P energy trading in the day-ahead (DA) stage, followed by adjustments of flexible resources and flexibility sharing through a receding-horizon feedback mechanism inspired by model predictive control (MPC) in the real-time (RT) stage. High-risk scenarios and computational overhead are managed by employing conditional value-at-risk (CVaR) measures, as well as approximation of constraints through dual robust reformulation and linear decision rules. To uphold the autonomy and privacy of prosumers, we propose distributed algorithms based on the fast consensus alternating direction method of multipliers (ADMM). Numerical results demonstrate that our framework achieves a 20% reduction in social cost while providing a coherent and interactive P2P energy trading environment, facilitating reliable network operation under high penetration of renewable energy sources without third-party intervention.

Federated Multiple Tensor-on-Tensor Regression (FedMTOT) for Multimodal Data under Data-Sharing Constraints

In recent years, diversified measurements reflect the system dynamics from a more comprehensive perspective in system modeling and analysis, such as scalars, waveform signals, images, and structured point clouds. To handle such multimodal structured high-dimensional (SHD) data, combining a large amount of data from multiple sites is necessary (i) to reduce the inherent population bias from a single site and (ii) to increase the model accuracy. However, impeded by data management policies and storage costs, data could not be easily shared or directly exchanged among different sites. Instead of simplifying or facilitating the data query process, we propose a federated multiple tensor-on-tensor regression (FedMTOT) framework to train the individual system model locally using (i) its own data and (ii) data features (not data itself) from other sites. Specifically, federated computation is executed based on alternating direction method of multipliers (ADMM) to satisfy data-sharing requirements, while the individual model at each site can still benefit from feature knowledge from other sites to improve its own model accuracy. Finally, two simulations and two case studies validate the superiority of the proposed FedMTOT framework.

Optimizing ADMM and Over-Relaxed ADMM Parameters for Linear Quadratic Problems

The Alternating Direction Method of Multipliers (ADMM) has gained significant attention across a broad spectrum of machine learning applications. Incorporating the over-relaxation technique shows potential for enhancing the convergence rate of ADMM. However, determining optimal algorithmic parameters, including both the associated penalty and relaxation parameters, often relies on empirical approaches tailored to specific problem domains and contextual scenarios. Incorrect parameter selection can significantly hinder ADMM's convergence rate. To address this challenge, in this paper we first propose a general approach to optimize the value of penalty parameter, followed by a novel closed-form formula to compute the optimal relaxation parameter in the context of linear quadratic problems (LQPs). We then experimentally validate our parameter selection methods through random instantiations and diverse imaging applications, encompassing diffeomorphic image registration, image deblurring, and MRI reconstruction.

Towards Multi-Mode Outlier Robust Tensor Ring Decomposition

Conventional Outlier Robust Tensor Decomposition (ORTD) approaches generally represent sparse outlier corruption within a specific mode. However, such an assumption, which may hold for matrices, proves inadequate when applied to high-order tensors. In the tensor domain, the outliers are prone to be corrupted in multiple modes simultaneously. Addressing this limitation, this study proposes a novel ORTD approach by recovering low-rank tensors contaminated by outliers spanning multiple modes. In particular, we conceptualize outliers within high-order tensors as latent tensor group sparsity by decomposing the corrupted tensor into a sum of multiple latent components, where each latent component is exclusive to outliers within a particular direction. Thus, it can effectively mitigate the outlier corruptions prevalent in high-order tensors across multiple modes. To theoretically guarantee recovery performance, we rigorously analyze a non-asymptotic upper bound of the estimation error for the proposed ORTD approach. In the optimization process, we develop an efficient alternate direction method of multipliers (ADMM) algorithm. Empirical validation of the approach's efficacy is undertaken through comprehensive experimentation.

Robust Blind Text Image Deblurring via Maximum Consensus Framework

The blind text image deblurring problem presents a formidable challenge, requiring the recovery of a clean and sharp text image from a blurry version with an unknown blur kernel. Sparsity-based strategies have demonstrated their efficacy by emphasizing the sparse priors of the latent image and kernel. However, these existing strategies have largely neglected the influence of additional noise, imposing limitations on their performance. To overcome this limitation, we propose a novel framework designed to effectively mitigate the impact of extensive noise prevalent in blurred images. Our approach centers around a robust Maximum Consensus Framework, wherein we optimize the quantity of interest from the noisy blurry image based on the maximum consensus criterion. Furthermore, we propose the integration of the Alternating Direction Method of Multipliers (ADMM) and the Half-Quadratic Splitting (HQS) method to address the computationally intractable L0 norm problem. This innovative strategy enables improvements in the deblurring performance of blurry text images with the additional synthetic noise. Experimental evaluations conducted on various noisy blurry text images demonstrate the superiority of the proposed approach over existing methods.